This paper introduces a new biologically-inspired training method named Continual Learning through Adjustment Suppression and Sparsity Promotion (CLASSP). CLASSP is based on two main principles observed in neuroscience, particularly in the context of synaptic transmission and Long-Term Potentiation (LTP). The first principle is a decay rate over the weight adjustment, which is implemented as a generalization of the AdaGrad optimization algorithm. This means that weights that have received many updates should have lower learning rates as they likely encode important information about previously seen data. However, this principle results in a diffuse distribution of updates throughout the model, as it promotes updates for weights that haven't been previously updated, while a sparse update distribution is preferred to leave weights unassigned for future tasks. Therefore, the second principle introduces a threshold on the loss gradient. This promotes sparse learning by updating a weight only if the loss gradient with respect to that weight is above a certain threshold, i.e. only updating weights with a significant impact on the current loss. Both principles reflect phenomena observed in LTP, where a threshold effect and a gradual saturation of potentiation have been observed. CLASSP is implemented in a Python/PyTorch class, making it applicable to any model. When compared with Elastic Weight Consolidation (EWC) using Computer Vision and sentiment analysis datasets, CLASSP demonstrates superior performance in terms of accuracy and memory footprint.

For most organisms, there are dangerous objects where even a close encounter with the object could be detrimental. A visual system that helps avoid close approaches with such objects enhances survival probability beyond what is afforded by one that just facilitates simple collision avoidance. However, “dangerous object” is a category that only has meaning in a particular context, and therefore recognizing them is a very complex task. Our objective is to determine how people find dangerous objects in a continuous stream of imagery, and develop a computational implementation of the model that can be tested on imagery. Evidence is cited suggesting that the visual pathways in higher animals implement a Composable Codebook that carries out object recognition. An internal, view-independent world model stores several different types of relational information that make it possible to fill-in incomplete objects and activities once the imagery is registered to the internal world model. Interneurons play a key role in all of the filling-in processes. We illustrate how the models of the visual pathway used by Stephen Grossberg and his group to separate textures can also mediate the registration of an internal world model to visual input.